Intra-video ratings

ABSTRACT

A system and method is provided for receiving and displaying voting data with respect to online hosted videos. Viewers vote on portions within various videos stored in an online video hosting website. Voting data can represent different types of votes, such as unipolar, bipolar, or categorical. The prior voting data, including that submitted by other viewers, can be displayed in a number of ways, such as gradations of color, or charts such as bar graphs. The voting data may be associated with demographic categories, such as a voter&#39;s age, sex, or political affiliation. Further, a video may be selectively played to display portions of most interest to given demographic categories, as indicated by voting patterns of those categories.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.12/414,553, filed on Mar. 30, 2009, which application is incorporatedherein by reference.

TECHNICAL FIELD

The present invention generally relates to the management and display ofvotes associated with video.

BACKGROUND

The proliferation of video sharing through websites provides numerousopportunities for users to collaborate and experience videos in onlinecommunities. Video hosting websites allow users to upload, view, andrate videos. Users browsing a video hosting website can locate videos ofinterest by, for example, searching for videos, browsing directories, orsorting by ratings.

User rating of videos provides a way to complement video with usefulinformation. However, because conventional systems allow user ratingonly of a video as a whole, it cannot be determined which segments ofthe video the users found particularly interesting. This is especiallyproblematic in light of increasingly large video length maximums allowedby video sharing websites, which leads to the presence of long videos,only a small portion of which may be interesting to most users. Inaddition, there is no way to navigate videos to find the most popularsegments of a video. Although a user may wish to view only the mostpopular segments of the video, the user is obliged to view the entirevideo because the most popular parts of the video have not beendetermined. This results in considerable waste of time and userfrustration.

SUMMARY

The present invention includes systems and methods for receiving anddisplaying voting data for a plurality of viewers of online videoshosted in a video hosting service. In one embodiment, a video hosted onthe service is provided to the viewers via their client devices. Theviewers vote as to their opinion on segments of the videos as they arebeing displayed on the viewers' devices. For example, the viewers canvote on whether they agree or disagree with the content of each segment.Each voter thus provides a series of votes corresponding to the segmentsof the video. For example, if a 1 minute video has 10 six-secondsegments, each viewer may cast 10 votes; or the video may have 60one-second segments in which case a viewer can cast sixty votes.Alternatively, segments in a video can have variable length, with somesegments being longer or shorter than other segments.

Each vote has a value that is selected from a unipolar or multipolar setof potential values, such as bipolar (e.g., agree/disagree,favorable/unfavorable, funny/not funny), or categorical votes.

The votes are transmitted from the client devices and received by thevideo hosting service. For each segment of the video, the votes can beaggregated according to their vote values; this aggregation ispreferably, though not necessarily, done while the video is beingprovided to the client devices. For example, where the viewers canprovide either a favorable or unfavorable vote, the number of favorableand unfavorable votes for each segment can be determined. The aggregatedinformation reveals the aggregate opinion or judgment of the viewers foreach segment of the video, and thus how that opinion may have changedthroughout the video.

The video hosting service transmits information about the aggregatedvotes for the previously displayed segments, such as a graphical ornumerical representation (e.g., totals for each vote value), to theviewer's client device, where it can be displayed in a number of ways,such as gradations of color, or charts such as bar graphs. The displayof the aggregated information is preferably done while the video isstill playing. Thus, as the viewer is watching the video and voting, shealso sees the aggregated voting information for segments of the videothat have already been shown. In one embodiment, the aggregatedinformation is displayed on a timeline representation.

The voting data may be associated with demographic categories, such as aviewer's age, sex, or political affiliation; the votes can be aggregatedaccording those demographic categories, and information about thedemographic aggregation can be provided to the client devices fordisplay thereon. Further, a video may be selectively played to displaysegments of most interest to given demographic categories, as indicatedby voting patterns of those categories.

In one embodiment, a computer-implemented method of associating a seriesof votes with a video, wherein the video has a plurality of segments,comprises transmitting the video for display to a client device of aviewer, whereby the client device plays back the video for viewing bythe viewer, and receiving a first series of votes from the viewer whilethe video is being played back on the client device, each voteassociated with one of the segments of the video and having a votevalue, the vote value selected by the viewer from a multipolar set ofpotential values. The method further comprises storing the first seriesof votes in a storage medium in association with information identifyingthe video.

The features and advantages described in this summary and the followingdetailed description are not all-inclusive. Many additional features andadvantages will be apparent to one of ordinary skill in the art in viewof the drawings, specification, and claims presented herein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a system architecture for receiving anddisplaying voting data associated with online hosted videos inaccordance with one embodiment of the present invention.

FIG. 2 illustrates steps performed by a client related to voting onvideos, according to one embodiment.

FIGS. 3A-3G illustrate user interfaces for use with voting on videos,according to different embodiments.

The figures depict various embodiments of the present invention forpurposes of illustration only. One skilled in the art will readilyrecognize from the following discussion that alternative embodiments ofthe structures and methods illustrated herein may be employed withoutdeparting from the principles of the invention described herein.

DESCRIPTION OF EMBODIMENTS

FIG. 1 is a block diagram of a system architecture in accordance withone embodiment. As illustrated in FIG. 1, a video hosting website 108includes a front end server 124, a video server 126, a network interface122, a video database 128, a video access log 129, a user database 140,a vote database 142, a vote display module 118, and a vote storagemodule 144. Other conventional features, such as firewalls, loadbalancers, application servers, failover servers, site management tools,and so forth are not shown so as to more clearly illustrate the featuresof the system. Examples of a suitable website 108 for implementation ofthe system include the YouTube™ and Google Video™ websites; other videohosting sites are known as well, and can be adapted to operate accordingthe teaching disclosed herein. It will be understood that the term“website” represents any system and method of providing content and isnot intended to be limited to systems that support content provided viathe Internet or the HTTP protocol. The various servers areconventionally implemented, whether as a single piece of software orhardware or as multiple pieces of software or hardware and can couple tothe network 105 via the network interface 122. In general, functionsdescribed in one embodiment as being performed on the server side canalso be performed on the client side in other embodiments ifappropriate.

A client 130 executes a browser 132, and connects to the front endserver 124 via a network 105, which is typically the Internet, but mayalso be any network, including but not limited to a LAN, a MAN, a WAN, amobile, wired or wireless network, a private network, or a virtualprivate network. While only a single client 130 and browser 132 areshown, it is understood that very large numbers (e.g., millions) ofclients are supported and can be in communication with the website 108at any time. The client 130 may include a variety of different computingdevices. Examples of client devices 130 are personal computers, digitalassistants, personal digital assistants, cellular phones, mobile phones,smart phones or laptop computers. As will be obvious to one of ordinaryskill in the art, the present invention is not limited to the deviceslisted above.

In some embodiments, the browser 132 includes an embedded video player134 such as, for example, the Flash™ player from Adobe Systems, Inc. orany other player adapted for the video file formats used in the videohosting website 108. A user can access a video from the video hostingwebsite 108 by browsing a catalog of videos, conducting searches onkeywords, reviewing play lists from other users or the systemadministrator (e.g., collections of videos forming channels), or viewingvideos associated with particular user group (e.g., communities).

Video server 126 receives uploaded media content, such as videos, fromcontent providers and allows content to be viewed by the client 130.Content providers may be users, commercial enterprises, and the like.Content may be uploaded to video server 126 via the Internet from apersonal computer, through a cellular network from a telephone or PDA,or by other means for transferring data over network 105 known to thoseof ordinary skill in the art. Content may be downloaded from videoserver 126 in a similar manner; in one embodiment media content isprovided as a file download to a client 130; in an alternativeembodiment, media content is streamed to client 130. The means by whichmedia content is received by video server 126 need not match the meansby which it is delivered to client 130. For example, a content providermay upload a video via a browser on a personal computer, whereas client130 may view that video as a stream sent to a PDA. Note also that videoserver 126 may itself serve as the content provider.

Users of clients 130 can also search for videos based on keywords, tagsor other metadata. These requests are received as queries by the frontend server 124 and provided to the video server 126, which isresponsible for searching the video database 128 for videos that satisfythe user queries. The video server 126 supports searching on any fieldeddata for a video, including its title, description, tags, author,category and so forth. User can also browse for videos of interest viacategories, channels, lists or other means.

Users of the clients 130 and browser 132 can upload content to the videohosting website 108 via network 105. The uploaded content can include,for example, video, audio or a combination of video and audio. Theuploaded content is processed and stored in the video database 128. Thisprocessing can include format conversion (transcoding), compression,metadata tagging, and other data processing. An uploaded content file isassociated with the uploading user, and so the user's account record isupdated in the user database 140 as needed. Each uploaded video isassigned a video identifier when it is processed.

For purposes of convenience and the description of one embodiment, theuploaded content will be referred to as “videos”, “video files”, or“video items”, but no limitation on the types of content that can beuploaded are intended by this terminology.

The video database 128 is used to store the received videos. The videodatabase 128 stores video content and associated metadata, provided bytheir respective content owners. The video files have metadataassociated with each file such as a video ID, artist, video title,label, genre, and time length. The video file data may be viewed ascomprising a number of video segments with which votes may beassociated. For example, in one embodiment segments are coherentportions of the video, e.g. a 5-second clip in a video of a debate of acandidate from a particular viewing angle, as determined by a videosegmentation algorithm. The segmentation could be determined based oncharacteristics of the video data itself, such as scene changes, or onvoting data, such as that created by clustering votes nearby in time tofind the boundaries of the various segments. Since a segment's start andend points can be determined based on, e.g., scene changes, the varioussegments can have variable lengths with respect to each other. Inanother embodiment, a video can be segmented into a set of fixed-lengthsegments, such as 1-second segments, or 10-second segments, orindividual frames.

Information about the votes received is stored in the user database 140.The user database 140 is responsible for maintaining a record of allusers viewing videos on the website. Each individual user is assigned auser ID. The user ID can be based on any identifying information, suchas the user's IP address, user name, or the like. The user database mayalso contain information about the reputation of the user in the videocontext, as well as through other applications, such as the use of emailor text messaging.

A video access log 129 within video database 128 stores each instance ofvideo access. Each entry in the access log 129 identifies a video beingaccessed, a time of access, an IP address of the user, a user ID ifavailable, cookies, search queries, data identifying the type ofinteraction with the video, and the time of every interaction with thevideo. Interaction types can include any user interactions in the userinterface of the website 108, such as playing, pausing, rewinding,forwarding and submitting votes or ratings for a video.

A vote database 142 stores votes entered by users for the various videosof the video database 128. Votes may be represented in the vote database142 as a vote value associated with a segment or segments of a video,such as a single frame or segment. Vote values may comprise varioustypes of data in different embodiments, representing the different typesof votes and what data is tracked. In one embodiment, a vote value typeis “unipolar,” merely representing a single concept, such as interest orfavorability, and thus need have no associated value. In otherembodiments, a vote value type is “multipolar,” i.e. having a pluralityof potential values. One type of multipolar vote is “bipolar,”representing two distinct possibilities, such as “like”/“dislike”,“favorable”/“unfavorable”, “positive”/“negative”, “interesting”/“notinteresting”, and the like. Another type of multipolar vote is a“categorical” vote, in which, a vote can take on one of a number ofvalues for a given category; as one example, a vote describing theuser's subjective emotional reaction to the video could have possiblevalues representing reactions such as “clever”, “boring”, “interesting”,“inspiring”, “frightening” and the like. In general, a multipolar votemay represent data from any one of a number of different domains, suchas a set of integers, or a continuous range of real numbers. The valuesmay be orderable, such as votes representing levels of favorability,such as “poor”, “OK”, “good”, and “great”, or they may not be orderable,such as categories representing different unrelated emotions such as“clever”, “inspiring”, “frightening”, etc.

Votes also have an associated temporal indicator indicating when thevote was submitted, which allows the derivation of statistics onpatterns of voting over time within a given video. The temporalindicator may take different forms in different embodiments. Forexample, in one embodiment the temporal indicator indicates a particulartime at which the vote was specified, e.g. a number of seconds into thevideo. In this case, the video hosting website 108 can map this time toa particular segment, through either a function or a stored map thatidentifies the times at which the segment boundaries of the video occur.

In another embodiment, the temporal indicator instead is an index of asegment; for example, for an embodiment having fixed length segments of10 seconds, votes specified 45 seconds and 108 seconds into the videocould be given zero-based indices of 5 and 11, indicating that they arein the 5th and 11th of the 10-second segments, respectively. In oneembodiment, the calculation of the index is performed by the client atthe time the vote is specified, e.g. by performing integer division ofthe second offset into the video by the fixed number of seconds persegment.

The temporal indicators can be specified as absolute values relative tothe start of the video as described above, or can be relative offsetsfrom the previous indicator; with the first temporal indicator being anoffset from the beginning of the video, either in time (e.g.,milliseconds) or frames, and then each following temporal indicatorbeing the amount of time or number of frames from the prior indicator.

Regardless of which type of temporal identifier is provided to the videohosting website 108, the server can use it to determine a particularsegment with which the vote is associated.

Votes may also have associated data such as demographic categoriesassociated with the user who submitted the vote, e.g., gender, agegroup, political party affiliation, or other categories of interest.This information allows for even finer grained analysis of votingpatterns for a given video.

A vote storage module 144 stores vote data provided by the client withinthe vote database 142. The vote storage module 144 can additionallystore vote-related information, such as a vote timestamp, vote totalsfor a given segment of a video, and the like.

FIG. 2 illustrates steps performed by the client 130 of FIG. 1. At onepoint, the client 130 displays 210 a video. While the video is beingdisplayed, the client 130 receives 220 voting data from the user asinputs to the client 130. Client 130 also displays 230 the receivedvoting data in association with the video. Note that these steps neednot be performed in a linear sequence. For example, prior votes for avideo have typically been received 220 by the time that the video isdisplayed 210. Further, any prior voting data may be displayed 230before the receipt of additional voting data at step 220. These stepsare now discussed in further detail in conjunction with FIGS. 3A-3G,which illustrate different aspects of user interfaces for receiving anddisplaying voting data according to various embodiments.

Referring again to FIG. 2, a video is displayed 210. The video server126 of the video hosting website 108 retrieves data for the video fromthe video database 128 and provides it (e.g., via streaming) to theclient 130, which then displays it, e.g. via the embedded player 134.Also displayed are controls for playback of the video, and alsooptionally for specifying voting data. For example, FIG. 3A illustratesone sample user interface for entering unipolar voting data. A video isdisplayed in video area 305, and an associated timeline 310 correspondsto the different segments of the video over time, with a marker 315indicating the current position of the video being displayed. Ingeneral, the timeline 310 provides a representation of votes over time,using visual indications such as colors, bars, and the like positionedat locations along the timeline to illustrate the vote values. In FIG.3A, the timeline 310 displays color gradients, such as gradient 325, todepict voting information from prior votes. The changes in color(depicted in black and white in FIG. 3A as shades of gray of greater orlesser intensity) show changes in voting across the different videosegments, e.g. showing a change in total number of votes for eachsegment. The displayed prior votes may be only those of the viewercurrently viewing and voting on the video, or they may include thosehistorical votes previously submitted by other viewers before the viewerbegan to view the video (e.g. votes submitted in the previous 30 days,or some other set time frame), and/or they may include votes of otherviewers who are watching the same video as the viewer, eithersynchronously (e.g., all viewers are watching a broadcast) orasynchronously (different viewers watching the video but starting atdifferent times), or any combination of the foregoing (e.g., showing theaggregate of historical and current synchronous votes).

Referring back to FIG. 2, while the video is being displayed, votingdata is received 220 at the client 130 from the user. (Such voting datamay be in addition to voting data received at a previous time.) Votingcan be done in real time while the video is being displayed, i.e. theuser inputs a series of votes for a video on the client 130substantially contemporaneously while the video is being played, so thatthe user's votes are received at substantially the same time that theuser is making a judgment regarding the video content, rather than theuser being obliged to halt playback of the video to vote, and/or tospecify only a single vote representing an overall impression of thevideo content. Further, in one embodiment, each vote is received andstored at the video hosting website 108 substantially immediately afterif it is entered at the client, including typical delays associated withtransmissions over the network 105. Alternatively, the votes can befirst cached at the client 130, and then transmitted in batch to thewebsite 108, for example when the user finishes watching the video.

In FIG. 3A, for example, voting icon 320 may both provide a visualdescription of the types of votes associated with the video, and alsoserve as a means of entering votes. For example, the voting icon 320indicates that the votes are unipolar (i.e., that there is only one typeof vote, and that it is favorable, as illustrated with the thumbs-upimage). Further, clicking on the voting icon 320 specifies a unipolarvote associated with a segment of the video corresponding to the marker315 at the time the vote is input.

Votes can be submitted in a number of different manners in differentembodiments. For example, a user could submit a vote using a graphicaluser interface provided by, for example, a web browser, such as clickingon voting icon 320 of FIG. 3A. Similarly, the user could click on apoint on the timeline 310 to designate a unipolar vote for the segmentof the video corresponding to the clicked point. Users may also clickand drag on the timeline 310 to specify a unipolar vote for a longerinterval of video, possibly comprising several distinct segments of thevideo. Users may also submit votes via a typical input device, such as akeyboard, where the press of a particular key corresponds to a givenvote value. For example, a user could specify a unipolar vote bypressing any key, or a bipolar or categorical vote by pressing a keyassociated with an accepted value (e.g. “f” or “u”, or “+” or “−”, or upand down arrows for favorable or unfavorable values of a bipolar vote,or “b”, “i”, or “s” to designate “boring”, “interesting”, or“stimulating” in a categorical vote). The vote value is then associatedwith a time corresponding to the position of the marker 315.

After a user has submitted voting data, the client 130 then communicatesthe voting data to the vote storage module 144 of the video hostingwebsite 108, which converts any raw values received into theircorresponding semantic values and stores them in the vote database 142.A raw value is the data associated with input action, such as the value“f” of a keypress, and a semantic value describes the meaning associatedwith the raw value, such as a bipolar vote specifying a favorable value.Alternatively, the raw values can be converted to semantic values on theclient 130 prior to being transmitted to the video hosting website 108.

Voting information specified by other users may or may not be displayedto the user at the time that the user is entering his or her own votes.Displaying votes of other users provides the user with a maximum ofinformation, but runs the risk of introducing bias, leading the user toemulate the votes of the other users. If the potential for bias is aconcern, the vote display module 118 can refrain altogether fromdisplaying the data from other users in contexts when the user can vote,or it can display the voting data from other users only at times beforethe currently playing segment of the video. For example, the userinterface of FIG. 3B displays only voting data in timeline 310 of otherusers associated with times prior to the current play time in the video,as indicated by video playback marker 315. The vote storage module 144may additionally store whether or not a given vote was specified whenthe voting data of other users was available; such information could beused to optionally filter out possibly biased data when displayingvoting data, or it could be analyzed to estimate the effects of bias onvoting.

In one embodiment, the vote storage module 144 additionally associates atemporal indicator, such as a timestamp or a segment index, with thevote. This temporal indicator allows determining—directly orindirectly—the segment to which the vote corresponds. With regard todetermining a segment index based on a timestamp, note that thetimestamp corresponding to the time that the vote was specified on theclient 130, or received by the video hosting website 108, need notcorrespond to the precise point in the video that prompted the user tosubmit the vote. For example, there will be some delay in the user'smental processes between the time at which the point of interest wasplayed and when the user decides to vote on it, and some delay in theuser's physical actions between the time that the user decides to voteon it and when he or she actually submits the vote, and these delayswill vary among different users. However, as long as there is a largeamount of vote data available, the vote storage module 144 can applyclustering algorithms to the votes, grouping votes nearby in time into asingle common group, and then identifying a segment of the videoassociated with that group. For example, the vote storage module 144could calculate an average vote time of the votes in the group, thenadjust that time by subtracting a known estimate of the reaction time ofthe average user, e.g. 1 second. Each vote in the group is thenassociated with the segment containing that adjusted time, e.g. byassociating the vote with the segment index of the segment.

The vote storage module 144 may also derive and store additional data inassociation with videos and/or segments of a given video. For example,the vote storage module 144 may compute a total number of votes for aparticular segment of a video and store it in association with thatsegment, updating this stored total when additional votes are specifiedfor that segment by additional users who watch the video and vote onthat segment. Alternatively, the vote storage module 144 may compute atotal value of the votes for a particular segment, rather than a rawcount of the number of votes for that segment. For example, weights maybe assigned to votes based on, for example, a reputation score of theuser, the reputation score being based on factors such as whether theuser is new or unrecognized, the usefulness of votes previouslysubmitted by that user as rated by other users, or other informationabout the user within the user database 140.

Since multiple copies of a video may exist in the video database 128,each of which could receive separate voting data, voting data can becomefragmented across different copies of the video, with some users viewingand voting on one particular copy, and other users viewing and voting onanother copy. Thus, in one embodiment the vote storage module 144additionally identifies any other copies of the same video that existwithin the video database 128 and aggregates the voting data for thevarious copies of the video into a single set of data for that video.The vote storage module 144 can accomplish the identification of othercopies of the video by a variety of means, such as by computing asimilarity score between a set of representative feature vectors of eachvideo. Each of the copies of the video is then associated with thisaggregate vote data.

Each vote can also be associated with a set of demographic attributes,such as age, sex, political affiliation, and the like. In oneembodiment, the demographic attributes are specified via a questionnaireprovided along with the video, such as that depicted in FIG. 3C, whichdepicts a questionnaire related to a political video. In FIG. 3C,selection menu 360 allows the user to specify one of a set of politicalparties, and selection menu 365 allows the user to specify a particularcandidate. Profile update checkboxes 370 allow the user to specify thatthe specified answers should be reflected in the user's profile on thevideo hosting website 108. Since the questionnaire is video-specific andcan be created by the owner of the video, as well as by the videohosting website 108 itself, the questionnaire should include some datathat specifies how the particular questions of the questionnairecorrespond to the fields of the global profile of the video hostingwebsite. Thus, the questionnaire may encode, for each field to beupdated, an identifier that is standard for profiles on the videohosting website 108. For example, the update checkbox 370 for thepolitical party selection menu 360 encodes an identifier specifying thatthe political party field of the user profile should be updated.Checkbox 370 may be omitted for questions not tied to a field of theuser profile. The questionnaire could be part of the video itself,including interactive elements for specifying and providing input, or itcould be a separate interactive multimedia unit that is linked to by theembedded video player 134 on the client 130 if the user provided anyvotes.

In another embodiment, the demographic attributes need not be explicitlyspecified by the user as in the interface of FIG. 3C, but can beautomatically extracted from the profile of the user if the user has aprofile that specifies values for the attributes in question. Forexample, if the user profile specifies a value of “Independent” for a“political party affiliation” attribute, then that value could beautomatically set in selection menu 360 of FIG. 3C, for example, or theinterface of FIG. 3C could alternatively be omitted entirely.

Regardless of the source of the attributes, the vote storage module 144associates each vote received with respect to a video with the provideddemographic attributes. For example, if the system-wide identity of theuser—such as a user ID—is known, the votes can simply be associated witha system-wide entry for that user identity, the entry comprising thespecified demographic attributes or the user ID for subsequent retrievalof the attributes. Alternatively, if no system-wide identity of the useris known, the votes by that user can still be grouped and stored inassociation with that particular video and with the specifieddemographic attributes.

Referring again back to FIG. 2, voting data is displayed 230. In FIG.3A, for example, the timeline 310 may be graphically depicted toillustrate the voting data associated with the video, as is furtherdescribed below; for example, the timeline of FIG. 3A contains colorgradations indicating different numbers of votes for different segments.

The vote display module 118 provides for the display of an indication ofpreviously stored voting data on a timeline associated with the videos.This provides a user with a visual summary of how users voted across thevarious segments of a video. The voting values can be displayed indifferent ways in different embodiments. For example, for unipolarvotes, times corresponding to the different segments of the video can becolored differently, such as by associating a first color with a maximumvote value and a second color with a minimum vote value and computing alinear blend of the colors for a given segment of the video based on thevote value of that segment. For instance, a segment with no associatedvotes could be colored yellow, and a segment having a number (or value)of votes at or exceeding the maximum value could be colored red, and asegment with a vote value between those extremes could be colored anappropriate blend of yellow and red. This is depicted in grayscale formin the timeline 310 of FIG. 3A, where darker shades represent the colorred, and lighter shades the color yellow.

In one embodiment, the display of the voting data, such as the timeline310, is updated in real time as votes are entered. For example, theuser's own votes for earlier segments of the video can be displayedsubstantially immediately after the votes are specified. Additionally,in some embodiments the votes of other users made at substantially thesame time, e.g. after the user began to view the user interface 300 forthe video, can be incorporated into the display. For example, the votesof other clients 130 for a given video can be stored in the votedatabase 142, and those votes, or updated graphical data reflectingthose votes, can then be provided to each of the other clients 130 thatis currently viewing that same video, and the display of the clients canbe updated accordingly.

In the case of bipolar voting, such as favorable versus unfavorablevotes, each can be assigned a color and the net value computed for agiven segment of the video. For example, referring back to FIG. 3B,color gradients 311-313 are indicated on the timeline 310 correspondingto segments where the voting is largely favorable, largely unfavorable,and a roughly equal, respectively. Although represented in grayscale inFIG. 3B, the colors for the gradients 311-313 represent the colorassociated with favorable icon 321, the color associated with theunfavorable icon 322, and a blend thereof, respectively. The color of aparticular color gradient can be a linear blend of the colorsrepresenting the number of favorable and unfavorable votes. For example,if favorable votes are represented by the color red, and unfavorablevotes by the color blue, then a segment with 60% favorable votes and 40%unfavorable votes could be represented with a linear blend having 60%red and 40% blue, resulting in a shade of purple. Alternatively, theoverall color could be that corresponding to the majority of the votes,and the saturation of the color could be based on how strong themajority is. For example, for a segment of the video for which there are150 red (favorable) votes and 100 blue (unfavorable) votes, the segmentcould be colored red, with the 50-vote difference between the total of250 favorable and unfavorable votes used to determine the intensity ofthe red. As an alternative example, votes can be represented with agraph, such as the bar charts depicted in FIG. 3D, which comprises onebar chart area 391 representing favorable votes over time, and anotherbar chart area 392 representing unfavorable votes over time, with longerbars representing more votes for the corresponding segments, and thehorizontal position of each bar corresponding to the approximate time ofthe votes contributing to the bar. FIG. 3D also depicts a button 393 forthe selection of particular demographics, e.g., filtering the votingdata to show just that voting data associated with users of a particularpolitical party.

In general, there can be video segments for which there are strongdifferences of opinion among the users viewing the video, such as alarge number of both favorable and unfavorable votes for the samesegment in the case of bipolar voting; this could indicate an area ofcontroversy or dispute. For example, this might occur in the context ofa video of a political debate, with Democratic viewers voting stronglyin favor of a particular segment of the video (e.g., in which aDemocratic candidate made a statement appealing to Democratic viewers),and Republican viewers voting strongly against it. In general,controversies are detected by quantifying a level of divergence amongthe votes for the segment and comparing the quantified divergence to athreshold value. In one embodiment for bipolar voting, such“controversial” segments are detected by computing the ratio offavorable to unfavorable votes, and determining the distance of thisratio from 0.5 (indicating split votes), with distances less than somespecified threshold distance (e.g., 0.1) indicating a controversy. Inother embodiments, it is additionally preliminarily determined whetherthere is a sufficient total number of votes (e.g., 100 votes); if not,then it is not deemed important enough to represent a controversy. Suchcontroversies can be difficult to visually detect for some types ofvisual display, such as color blending, where an equal number ofpositive and negative values produces the same blended color, regardlessof whether votes are unanimously neutral or strongly but equallydivided. Thus, in some embodiments the system can additionally provide avisual indicator to clearly identify such segments of controversy. Forexample, FIG. 3E depicts three star-shaped controversy indicators 394corresponding to three segments in which there are a large number ofboth positive and negative votes. Users can then move playback to thosecontroversial segments of the video, e.g. by clicking at those locationsor dragging the marker 315 to them.

Controversies may be shown between different demographic groups, e.g.all voters, as in FIG. 3E, or within a single group. For example, FIG.3F illustrates showing separate voting data and separate controversyindicators 394 on separate timelines 310 for each, e.g. within theRepublican, Independent, and Democratic groups.

The previously stored voting data can also be filtered and used to focuson segments of interest within the video. For example, FIG. 3G depicts auser interface for specifying watching a video segmented according to agiven demographic attribute or attributes, e.g., political party.Specifically, FIG. 3G shows selection lists 395 and 396 for selectingvotes by those of a particular party, and who mostly favor a particularcandidate, respectively. In response to, for instance, selection ofbutton 397, the selected attribute values are then applied as filters,selecting only votes associated with the selected demographic attributevalues. The identified votes are then displayed on the timeline 310,e.g., as a group visually distinct from votes not associated with theselected demographic attribute values. For example, only the votesassociated with the selected demographic attribute values may bedisplayed, or they may be displayed in addition to but separately fromother votes, such as on a separate timeline, as in FIG. 3F, where eachvalue of the political party demographic attribute (e.g., Republicans,Independents, Democrats) is depicted on a separate timeline.

Additionally, the client player 134 can be set to play only thosesegments of the video that are considered of interest to the selecteddemographics, e.g. those segments having particularly high votingvalues, or segments that are determined to be controversial. The userinterface of FIG. 3G could be shown at different times in differentembodiments, such as at the start of the video, at the end of the video,or in response to a user request to select segments of interest to givendemographics, e.g. via selection of the button 393 of FIG. 3D. In oneembodiment, the client 130 transmits a designation of the selecteddemographic categories to the video hosting website 108, which thendetermines which segments of the video have at least a threshold numberof votes that are associated with a demographic attribute value(s)matching the designated demographic attribute value(s), and thenprovides an indication of the determined segments (e.g., a segmentidentifier) to the client 130, which can then display only thosesegments. For example, if a viewer selected “Independent” as a politicalparty demographic attribute value, then the video hosting website 108would provide to the client 130 an indication of only the segments withat least a threshold number of votes associated with the Independentpolitical party, e.g. those votes by viewers with profiles specifyingthat party.

Reference in the specification to “one embodiment” or to “an embodiment”means that a particular feature, structure, or characteristic describedin connection with the embodiments is included in at least oneembodiment of the invention. The appearances of the phrase “in oneembodiment” in various places in the specification are not necessarilyall referring to the same embodiment.

It should be noted that the process steps and instructions of thepresent invention can be embodied in software, firmware or hardware, andwhen embodied in software, can be downloaded to reside on and beoperated from different platforms used by a variety of operatingsystems.

The present invention also relates to an apparatus for performing theoperations herein. This apparatus may be specially constructed for therequired purposes, or it may comprise a general-purpose computerselectively activated or reconfigured by a computer program stored inthe computer. Such a computer program may be stored in a computerreadable storage medium, such as, but is not limited to, any type ofdisk including floppy disks, optical disks, CD-ROMs, magnetic-opticaldisks, read-only memories (ROMs), random access memories (RAMs), EPROMs,EEPROMs, magnetic or optical cards, application specific integratedcircuits (ASICs), or any type of media suitable for storing electronicinstructions, and each coupled to a computer system bus. Furthermore,the computers referred to in the specification may include a singleprocessor or may be architectures employing multiple processor designsfor increased computing capability.

The algorithms and displays presented herein are not inherently relatedto any particular computer or other apparatus. Various general-purposesystems may also be used with programs in accordance with the teachingsherein, or it may prove convenient to construct more specializedapparatus to perform the required method steps. The required structurefor a variety of these systems will appear from the description below.In addition, the present invention is not described with reference toany particular programming language. It will be appreciated that avariety of programming languages may be used to implement the teachingsof the present invention as described herein, and any references belowto specific languages are provided for disclosure of enablement and bestmode of the present invention.

While the invention has been particularly shown and described withreference to a preferred embodiment and several alternate embodiments,it will be understood by persons skilled in the relevant art thatvarious changes in form and details can be made therein withoutdeparting from the spirit and scope of the invention.

Finally, it should be noted that the language used in the specificationhas been principally selected for readability and instructionalpurposes, and may not have been selected to delineate or circumscribethe inventive subject matter. Accordingly, the disclosure of the presentinvention is intended to be illustrative, but not limiting, of the scopeof the invention, which is set forth in the following claims.

What is claimed is:
 1. A computer-implemented method of associating aseries of votes with a video, the method comprising: transmitting thevideo for display to a client device of a viewer; receiving a firstseries of votes from the viewer, each vote having a vote value; storingthe first series of votes in a storage medium; receiving a second seriesof votes of other viewers; associating votes of the first series ofvotes and of the second series of votes with demographic attributevalues; selecting one of the demographic attribute values; identifying,from at least one of the first series of votes and the second series ofvotes, votes associated with the selected demographic attribute value;and providing, to the client device for display, a graphical timelinerepresentation comprising the vote values of the first series of votesand vote values of the second series of votes, wherein the votesassociated with the selected demographic attribute value are representedon the graphical timeline representation as a group visually distinctfrom votes not associated with the selected demographic attribute value.2. The computer-implemented method of claim 1, wherein the video has aplurality of segments, wherein the votes of the first series of votesand of the second series of votes are associated with ones of thesegments, and wherein providing the graphical timeline representationfurther comprises: identifying a controversial video segment byquantifying a level of divergence among the votes associated with thesegment and comparing the quantified level of divergence to a threshold;and providing to the client device, for display on the providedgraphical timeline representation at a location associated in time withthe controversial video segment, a visual indicator of the controversy.3. The computer-implemented method of claim 1, wherein the graphicaltimeline representation displays the vote values as gradations of color.4. The computer-implemented method of claim 1, wherein the graphicaltimeline representation displays the vote values as a graph.
 5. Thecomputer-implemented method of claim 1, wherein the second series ofvotes includes votes from other viewers watching the video synchronouslywith the viewer.
 6. The computer-implemented method of claim 1, whereinthe second series of votes includes votes from other viewers watchingthe video asynchronously with the viewer.
 7. The computer-implementedmethod of claim 1, wherein the second series of votes includeshistorical votes from other viewers having watched the video before theviewer began to view the video.
 8. The computer-implemented method ofclaim 1, wherein the video has a plurality of segments, and wherein thevotes of the first series of votes and of the second series of votes areassociated with ones of the segments, the method further comprising:responsive at least in part to receiving from the viewer a designationof a value of a demographic attribute: determining segments of the videohaving at least a threshold number of votes that are associated with ademographic attribute value matching the designated demographicattribute value; and providing, to a client device of the viewer, anindication of the determined segments of the video.
 9. A non-transitorycomputer-readable storage medium storing a computer program executableby a computer system for voting on a digital video, the computer programcontrolling the computer system to perform operations comprising:receiving the video; receiving, from the viewer, a series of votes, eachvote having a vote value; selecting a demographic attribute value;receiving from a vote database a second series of votes, the secondseries of votes being associated with the selected demographic attributevalue; and displaying a graphical timeline representation of thereceived votes and of the second series of votes, wherein the graphicaltimeline representation displays the second series of votes as a groupvisually distinct from votes not associated with the selecteddemographic attribute value.
 10. The computer-readable storage medium ofclaim 9, wherein the vote value is selected by the viewer from amultipolar set of potential values.
 11. The computer-readable storagemedium of claim 9, wherein the video has a plurality of segments andwherein the votes of the first series of votes and of the second seriesof votes are associated with ones of the segments, the operations of thecomputer program further comprising: receiving an indication of a videosegment having a quantified level of divergence among the votesassociated with the segment greater than a specified threshold; anddisplaying an indicator of a controversy at a location on the graphicaltimeline representation corresponding to the video segment for which theindication was received.
 12. The computer-readable storage medium ofclaim 9, wherein the graphical timeline representation displays the votevalues as gradations of color.
 13. The computer-readable storage mediumof claim 9, wherein the graphical timeline representation displays thevote values as a graph.
 14. The computer-readable storage medium ofclaim 9, wherein the video has a plurality of segments, and wherein thevotes of the first series of votes and of the second series of votes areassociated with ones of the segments, the operations of the computerprogram further comprising: receiving from the viewer a designation of avalue of a demographic attribute; responsive at least in part to thereceiving of the designation, displaying, within the video playbackarea, segments of the video having at least a threshold number of votesthat are associated with a demographic attribute value matching thedesignated demographic attribute value.
 15. A computer system forassociating a series of votes with a first video, the system comprising:a video database storing a plurality of videos including the firstvideo; a vote database storing votes associated with videos; a moduleconfigured to perform operations comprising: transmitting the firstvideo from the video database for display to a client device of aviewer; receiving a first series of votes from the viewer, each votehaving a vote value, the vote value selected by the viewer from amultipolar set of potential values; storing the first series of votes inthe vote database in association with information identifying the firstvideo; receiving a second series of votes of other viewers; associatingvotes of the first series of votes and of the second series of voteswith demographic attribute values; selecting one of the demographicattribute values; identifying, from at least one of the first and secondseries of votes, votes associated with the selected demographicattribute value; and providing, to the client device for display, agraphical timeline representation comprising the vote values of thefirst series of votes and vote values of the second series of votes,wherein the votes associated with the selected demographic attributevalue are represented on the graphical timeline representation as agroup visually distinct from votes not associated with the selecteddemographic attribute value.
 16. The computer system of claim 15,wherein the video has a plurality of segments, wherein the votes of thefirst series of votes and of the second series of votes are associatedwith ones of the segments, and wherein providing the graphical timelinerepresentation further comprises: identifying a video segment byquantifying a level of divergence among the votes associated with thesegment and comparing the quantified level of divergence to a threshold;and providing to the client device, for display on the providedgraphical timeline representation at a location associated in time withthe identified video segment, a visual indicator of a controversy. 17.The computer system of claim 15, wherein the graphical timelinerepresentation displays the vote values as gradations of color.
 18. Thecomputer system of claim 15, wherein the graphical timelinerepresentation displays the vote values as a graph.
 19. The computersystem of claim 15, the actions of the module further comprising:responsive at least in part to receiving from the viewer a designationof one of the demographic attribute values: determining segments of thevideo having at least a threshold number of votes that are associatedwith a demographic attribute value matching the designated demographicattribute value; and providing, to a client device of the viewer, anindication of the determined segments of the video.
 20. Acomputer-implemented method of displaying voting information for avideo, the video viewed by a plurality of viewers on a plurality ofrespective client devices, the video having a plurality of segments, themethod comprising: transmitting the video to the plurality of clientdevices of the plurality of viewers, whereby the video is displayed onthe client devices; receiving from each viewer's client device, whilethe video is being displayed on that viewer's client device, a series ofvotes from the viewer, each vote associated with a segment of the videoand having a vote value; for each segment of the video, aggregating thereceived votes associated with that segment, according to the votevalues, to form aggregated voting information; and providing, fordisplay to at least one of the plurality of client devices, a graphicaltimeline representation comprising the vote values of the series ofvotes and vote values of a second series of votes received from otherviewers, the providing comprising: associating votes of the first andsecond series of votes with demographic attribute values; responsive toreceiving a selection of one of the demographic attribute values:identifying, from the first and second series of votes, votes associatedwith the selected demographic attribute value; and providing, to theclient device for display, a representation of the identified votes onthe graphical timeline representation as a group visually distinct fromvotes not associated with the selected demographic attribute value. 21.The computer-implemented method of claim 20, further comprisingtransmitting the aggregated voting information to a plurality of clientdevices for display thereon concurrently with display of the video. 22.A computer-implemented method of displaying voting information for avideo, the video comprising a plurality of segments, each segmentassociated with aggregated voting information for votes specified byviewers, the votes having values selected by the viewers from amultipolar set of potential values, the votes being associated withdemographic attribute values, the aggregated voting informationcomprising a total number of votes for each potential value of themultipolar set, the method comprising: transmitting the video to aclient device of a viewer, whereby the segments of the video aredisplayed on the client device; and transmitting for display on theclient device concurrently with display of the video, the aggregatedvoting information for previously displayed segments of the video;responsive to receiving a selection of one of the demographic attributevalues, identifying votes associated with the selected demographicattribute value; and transmitting, to the client device for display, agraphical timeline representation comprising the vote values of thevotes, wherein the votes associated with the selected demographicattribute value are represented on the graphical timeline representationas a group visually distinct from votes not associated with the selecteddemographic attribute value.